In general, image enhancement involves applying one or more operations to an image to improve the image quality, for example, sharpening improves image details, noise reduction removes image noise, de-blocking removes blocking artifacts caused, for example, by JPEG image compression, scene balance adjustment improves brightness and color balance, and tone-scale adjustment improves image contrast and rendering.
While these methods do indeed produce enhanced images, the quality of the resulting image often varies depending on the image content. For example, using the unsharp mask algorithm may produce a pleasing result for an image of a building. However, using the same algorithm may result in the undesirable appearance of oversharpening for an image of a human face (e.g., wrinkles, blemishes may be unpleasantly “enhanced”, i.e., made more visible). For another example, using a smoothing algorithm helps remove the amount of noise and/or blocking artifacts and produce a pleasing result for an image of a human face or clear blue sky. However, the same operation of the same amount may result in undesirable removal of details in grass lawn, textured fabric, or animal hair. Conventionally, the amount of sharpening, or any other type of enhancement, needs to be adjusted individually for each scene by a human operator, an expensive process. Another drawback of the conventional approach is that the amount of sharpening cannot be adjusted easily on a region by region basis within the same image, resulting in having to apply an amount of enhancement that is a trade-off between different amounts required by different subject matters or objects in the scene.
In the prior art, there are examples of modifying an image enhancement operation based on pixel color. For example, in U.S. Pat. No. 5,682,443 issued Oct. 28, 1997, Gouch et al. describe a method of modifying, on a pixel by pixel basis, the parameters associated with the unsharp mask. Sharpening an image with unsharp masking can be described with the following equation:s(x,y)=i(x,y)**b(x,y)+βf(i(x,y)−i(x,y)**b(x,y))  (1)where:                s(x,y)=output image with enhanced sharpness        i(x,y)=original input image        b(x,y)=lowpass filter        β=unsharp mask scale factor        f( )=fringe function        ** denotes two dimensional convolution        (x,y) denotes the x-th row and the y-th column of an image        
Typically, an unsharp image is generated by convolution of the image with a lowpass filter (i.e., the unsharp image is given by i(x,y)**b(x,y)). Next, the highpass, or fringe data is generated by subtracting the unsharp image from the original image (the highpass data is given by i(x,y)−i(x,y)**b(x,y)). This highpass data is then modified by either a scale factor β or a fringe function f( ) or both. Finally, the modified highpass data is summed with either the original image or the unsharp image to produce a sharpened image.
Gouch et al. teach that the fringe function may be dependent on the color of the pixel i(x,y) This feature allows them to tailor the sharpening preformed for those pixels which are similar in color to flesh, for example. However, this method is not based on a probability or degree of belief that specific image pixels represent human flesh, and thus likely unnecessarily conservatively sharpens image regions having a similar color to human flesh such as bricks or wood. The method of Gouch et al. exclusively uses image color and does not allow for the use of other features such as texture or shape features which research has shown to effectively classify image regions.
Schwartz discloses the concept of automatic image correction using pattern recognition techniques in Europe Patent Application 0681268, filed Apr. 10, 1995, wherein the pattern recognition sub-system detects the presence and location of color-significant objects. Another similar method of selective enhancement of image data was described by Cannata, et al. in U.S. Ser. No. 09/728,365, filed Nov. 30, 2000, published Oct. 18, 2001. In one embodiment, spatial processing transformations such as sharpening are reduced or bypassed for pixels having color code values within a range known to be adversely affected by the spatial processing transformations. In another embodiment, color correction processing transformations are bypassed for pixels having color code values with a neutral color range. Clearly, since color is the only characteristic used to perform selective enhancement, undesirable effects can be obtained for other subject matters with similar colors.
Therefore, there exists a need for determining the types and amounts of enhancement for a particular image, whereby the local quality (e.g., sharpness and color) of the image can be improved depending on detecting different objects or subject matters contained within the image.